---
title: 'Replication Codes for ''***Camouflaged Propaganda: A Survey Experiment on
  Political Native Advertising***'''
author: "Yaoyao Dai and Luwei Luqiu"
output:
  html_document:
    df_print: paged
    theme: cerulean
    highlight: zenburn
    toc: yes
    toc_depth: 3
    toc_float:
      collapsed: no
      smooth_scroll: yes
  html_notebook:
    df_print: paged
    theme: cerulean
    toc: yes
    toc_depth: 3
    toc_float:
      collapsed: no
      smooth_scroll: yes
editor_options:
  chunk_output_type: inline
---

***
This is an R notebook with codes to replicate all the tables and figures in the paper *"Camouflaged Propaganda: A Survey Experiment on Political Native Advertising"* and its Online Appendices. To replicate the analysis, please download the replication data to the same folder as this .rmd file and run the .rmd file in R. You can also copy and paste the code chunks to a .r file and run the codes only. 

# Load data and packages
```{r results='hide', message=FALSE}
rm(list=ls(all=TRUE))
options(scipen=999)
library(foreign)
library(MASS)
library(stargazer)
library(car)
```

```{r}
PNA.data <- read.csv("PNA_data.csv", header=T)
```

## Codebook

Variable  | Variable Type | Values  
-|--|:----
G.Ewapo | Group Indicator |$1$: Primed Wapo Group;<br> $0$: Otherwise.
G.Wapo | Group Indicator |$1$: Wapo Group;<br> $0$: Otherwise.
G.Etele | Group Indicator |$1$: Primed Telegraph Group;<br> $0$: Otherwise.
G.Tele | Group Indicator |$1$: Telegrapg Group;<br> $0$: Otherwise.
G.CDw | Group Indicator |$1$: China Daily Article W Group;<br> $0$: Otherwise.
G.CDt | Group Indicator |$1$: China Daily Article T Group;<br> $0$: Otherwise.
Tr.WapoA | Treatment Indicator |$1$: Received Wapo China Watch article (Primed Wapo and Wapo Groups);<br> $0$: Otherwise (The other 4 groups).
Tr.TeleA | Treatment Indicator |$1$: Received Telegraph China Watch article (Primed Telegraph and Telegraph Groups);<br> $0$: Otherwise (The other 4 groups).
CorSour | DV: Detecting source | $1$: Identified source of artical as China Daily; <br> $0$: Otherwise.
PreTrust.Wapo | Pre-treatment Trust in The Washington Post | $1$: None at all; $2$: A little; $3$: Somewhat trust; <br> $4$: A Moderate amount; $5$: A lot; $6$: A great deal. 
PreTrust.Tele | Pre-treatment Trust in The Telegraph | $1$: None at all; $2$: A little; $3$: Somewhat trust; <br> $4$: A Moderate amount; $5$: A lot; $6$: A great deal.
PostTrust.Wapo | Post-treatment Trust in The Washington Post | $1$: None at all; $2$: A little; $3$: Somewhat trust; <br> $4$: A Moderate amount; $5$: A lot; $6$: A great deal;<br> NA: Respondents did not receive Wapo article or did not answer. 
PostTrust.Tele | Post-treatment Trust in The Telegraph | $1$: None at all; $2$: A little; $3$: Somewhat trust; <br> $4$: A Moderate amount; $5$: A lot; $6$: A great deal;<br> NA: Respondents did not receive Telegraph article or did not answer. 
Conv | DV: Level of Persuasiveness | $1$: Highly unconvincing; $2$: Unconvincing; $3$: Somewhat convincing;<br> $4$: Convincing; $5$: Highly convincing. 
Exp | CV: Media Experience| Additive measure based on 5 survey items with possible values from $5$ to $25$. Higher value means more media experience. Check Online Appendix D for more information on each individual item. 
Lit | CV: Media Literacy | Additive measure based on 14 survey items with possible values from $14$ to $70$. Higher value indicates greater media literacy. Check Online Appendix D for more information on each individual item.
TrustMed | CV: Trust in Media| Additive measure based on 7 survey items with possible values from $7$ to $35$. Higher value indicates greater trust in media in general. Check Online Appendix D for more information on each individual item.
Seconds | Quality Control: Duration of responses | How much time a respondent spent in completing the survey, measured in seconds.
Female | Demographic: Gender |$1$: Female;<br> $0$: Male.
Income | Demographic: Pre-tax household income |$1$: Less than $\$10,000$; $2$: $\$10,000$ to $\$19,999$; $3$: $\$20,000$ to $\$29,999$;<br> $4$: $\$30,000$ to $\$39,999$; $5$: $\$40,000$ to $\$49,999$; $6$: $\$50,000$ to $\$59,999$ <br> $7$: $\$60,000$ to $\$69,999$; $8$: $\$70,000$ to $\$79,999$; $9$: $\$80,000$ to $\$89,999$ <br> $10$: $\$90,000$ to $\$99,999$; $11$:$\$100,000$ to $\$149,999$; <br>$12$: $\$150,000$ or more.
Edu | Demographic: Level of Education | $1$: Less than high school degree;<br> $2$: High school graduate (high school diploma or equivalent including GED);<br> $3$: Some college but no degree; $4$: Associate degree in college (2-year);<br> $5$: Bachelor's degree in college (4-year); $6$: Master's degree;<br> $7$: Professional degree (JD, MD); $8$: Doctoral degree.
***

# Descriptive Statistics 

### Appendix A Table 1

```{r}
g.Ewapo <- subset(PNA.data, G.Ewapo==1) #Primed Wapo Group in Figure 1
g.Wapo <- subset(PNA.data, G.Wapo==1) # Wapo Group in Figure 1
g.Etele <- subset(PNA.data, G.Etele==1) # Primed Telegraph Group in Figure 1
g.Tele <- subset(PNA.data, G.Tele==1) # Telegrapg Group in Figure 1
g.CDw <- subset(PNA.data, G.CDw==1) # China Daily Article W in Figure 1
g.CDt <- subset(PNA.data, G.CDt==1) # China Daily Article T in Figure 1
```


```{r results='hide'} 
t.test(g.Wapo$Age,g.Ewapo$Age)
t.test(g.Etele$Age,g.Ewapo$Age)
t.test(g.Tele$Age,g.Ewapo$Age)
t.test(g.CDw$Age,g.Ewapo$Age)
t.test(g.CDt$Age,g.Ewapo$Age)
```

```{r results='hide'}
sd(g.Ewapo$Age)
sd(g.Wapo$Age)
sd(g.Etele$Age)
sd(g.Tele$Age)
sd(g.CDw$Age)
sd(g.CDt$Age)
```


```{r results='hide'}
t.test(g.Wapo$Female,g.Ewapo$Female)
t.test(g.Etele$Female,g.Ewapo$Female)
t.test(g.Tele$Female,g.Ewapo$Female)
t.test(g.CDw$Female,g.Ewapo$Female)
t.test(g.CDt$Female,g.Ewapo$Female)
```

```{r results='hide'}
sd(g.Ewapo$Female)
sd(g.Wapo$Female)
sd(g.Etele$Female)
sd(g.Tele$Female)
sd(g.CDw$Female)
sd(g.CDt$Female)
```

```{r results='hide'}
t.test(g.Wapo$Edu,g.Ewapo$Edu)
t.test(g.Etele$Edu,g.Wapo$Edu)
t.test(g.Tele$Edu,g.Wapo$Edu)
t.test(g.CDw$Edu,g.Wapo$Edu)
t.test(g.CDt$Edu,g.Wapo$Edu)
```

```{r results='hide'}
sd(g.Ewapo$Edu)
sd(g.Wapo$Edu)
sd(g.Etele$Edu)
sd(g.Tele$Edu)
sd(g.CDw$Edu)
sd(g.CDt$Edu)
```

```{r results='hide'}
t.test(g.Wapo$Income,g.Ewapo$Income)
t.test(g.Etele$Income,g.Wapo$Income)
t.test(g.Tele$Income,g.Wapo$Income)
t.test(g.CDw$Income,g.Wapo$Income)
t.test(g.CDt$Income,g.Wapo$Income)
```

```{r results='hide'}
sd(g.Ewapo$Income)
sd(g.Wapo$Income)
sd(g.Etele$Income)
sd(g.Tele$Income)
sd(g.CDw$Income)
sd(g.CDt$Income)
```


# Detecting PNA

### Table 1
```{r}
Model.source <- glm(CorSour ~ Tr.WapoA + EduP + Age, data=PNA.data, family=binomial(link = "logit"))
Model.source.c <- glm(CorSour ~ Tr.WapoA + EduP + Exp + Lit + TrustMed + Age 
                      + Edu + Income + Female, data=PNA.data, family=binomial(link = "logit"))
```

```{r message=FALSE}
stargazer(Model.source, Model.source.c, dep.var.labels = "Correct Source", covariate.labels = 
            c("Washington Post","Education Priming", "Media Experience", "Media Literacy",
              "MediaTrust", "Age", "Education", "Income","Female","Constant"), type="text")
```

# Reputational Costs of PNA

### Table 2
```{r}
#Only those received a Political Native Advertising treatment are relevant to this analysis
treated.data <- PNA.data[c(which(PNA.data$Tr.WapoA==1),which(PNA.data$Tr.TeleA==1)),]
```

```{r results='hide'}
t.test(treated.data$PostTrust.Wapo,treated.data$PreTrust.Wapo,paired = T)
t.test(treated.data$PostTrust.Tele, treated.data$PreTrust.Tele,paired =T)
```


### Table 3

```{r results='hide'}
#Self-Detected 
t.test(treated.data$PostTrust.Wapo[which(treated.data$CorSour == 1)],
       treated.data$PreTrust.Wapo[which(treated.data$CorSour == 1)], paired =T)
#Not Self-Detected
t.test(treated.data$PostTrust.Wapo[which(treated.data$CorSour == 0)],
       treated.data$PreTrust.Wapo[which(treated.data$CorSour == 0)], paired =T)
#Self-Detected 
t.test(treated.data$PostTrust.Tele[which(treated.data$CorSour == 1)],
       treated.data$PreTrust.Tele[which(treated.data$CorSour == 1)], paired =T)
#Not Self-Detected
t.test(treated.data$PostTrust.Tele[which(treated.data$CorSour == 0)],
       treated.data$PreTrust.Tele[which(treated.data$CorSour == 0)], paired =T)
```

# Persuasiveness of PNA


### Table 4
```{r results='hide'}
# Washington Post China Watch article comparing to the same article on China Daily 
# Among respondents who did identify true source on their own  
t.test(g.Wapo[which(g.Wapo$CorSour==1),]$Conv, g.CDw$Conv) 
# Washington Post China Watch article comparing to the same article on China Daily 
# among respondents who did not identify true source on their own  
t.test(g.Wapo[which(g.Wapo$CorSour==0),]$Conv, g.CDw$Conv)
```



```{r results='hide'}
# The Telegraph China Watch article comparing to the same article on China Daily 
# Respondents did identify true source on their own  
t.test(g.Tele[which(g.Tele$CorSour==1),]$Conv, g.CDt$Conv)
# The Telegraph China Watch article comparing to the same article on China Daily
# Respondents did not identify true source on their own  
t.test(g.Tele[which(g.Tele$CorSour==0),]$Conv, g.CDt$Conv)

```

# Appendix 

## Quality of Treatment Articles

### Appendix A Table 2

```{r results='hide'}
t.test(g.CDw$Accu,g.CDt$Accu)
t.test(g.CDw$Bias,g.CDt$Bias)
t.test(g.CDw$Conv,g.CDt$Conv)
```

## Quality of Responses 

```{r}
long <- PNA.data[PNA.data$Seconds >= 60,] # Only include the attentive respondents
```


### Appendix A Table 3


```{r}
ml.source <- glm(CorSour ~ Tr.WapoA + EduP + Age, data=long, family=binomial(link = "logit"))
ml.source.c <- glm(CorSour ~ Tr.WapoA + EduP + Exp + Lit + TrustMed  
                      + Age + Edu + Income + Female, data=long, family=binomial(link = "logit"))
```

```{r message=FALSE}
stargazer(ml.source, ml.source.c, dep.var.labels = "Correct Source", covariate.labels = 
            c("Washington Post","Education Priming", "Media Experience","Media Literacy",
              "MediaTrust", "Age", "Education", "Income","Female","Constant"), type="text")
```

### Appendix A Table 4
```{r}
treated.ldata <- subset(treated.data, Seconds >=60)
```

```{r results='hide'}
t.test(treated.ldata$PostTrust.Wapo,treated.ldata$PreTrust.Wapo,paired = T)
t.test(treated.ldata$PostTrust.Tele, treated.ldata$PreTrust.Tele, paired =T)
```

### Appendix A Table 5

```{r}
lg.Ewapo <- subset(long, G.Ewapo==1)
lg.Wapo <- subset(long, G.Wapo==1)
lg.Etele <- subset(long, G.Etele==1)
lg.Tele <- subset(long, G.Tele==1)
lg.CDw <- subset(long, G.CDw==1)
lg.CDt <- subset(long, G.CDt==1)
```

```{r results='hide'}
t.test(lg.Wapo[which(lg.Wapo$CorSour==1),]$Conv, lg.CDw$Conv)
t.test(lg.Wapo[which(lg.Wapo$CorSour==0),]$Conv, lg.CDw$Conv)
```


```{r results='hide'}
t.test(lg.Tele[which(lg.Tele$CorSour==1),]$Conv, lg.CDt$Conv)
t.test(lg.Tele[which(lg.Tele$CorSour==0),]$Conv, lg.CDt$Conv)
```

## Additional Robustness Checks

### Appendix A Table 6

```{r}
# Create a single variable of post-pre difference in trust in hosting media outlet
treated.data$dif.trust.wapo <- treated.data$PostTrust.Wapo -  treated.data$PreTrust.Wapo 
treated.data$dif.trust.tele <- treated.data$PostTrust.Tele -  treated.data$PreTrust.Tele
treated.data$Dif.trustB <- recode(treated.data$dif.trust.wapo,"NA=0") + 
  recode(treated.data$dif.trust.tele,"NA=0") 

```


```{r}
# Create a single variable of pre-treatment trust in host media outlet 
# PreTrustB indicates a Wapo Group respondent's pre-treatment trust to Washington Post
# and a Telegraph Group respondent's pre-treatment trust to Telegraph
treated.data$PreTrustB <- replace(treated.data$PreTrust.Wapo, which(treated.data$Tr.TeleA==1),0) +
  replace(treated.data$PreTrust.Tele, which(treated.data$Tr.WapoA==1),0)
```

```{r}
model.diff.trustB <- lm(Dif.trustB ~ PreTrustB + CorSour ,data=treated.data)

model.diff.trustB.c <- lm(Dif.trustB ~ PreTrustB+CorSour + Age + Female + Edu 
                          + Income + TrustMed + Lit + Exp,data=treated.data)

model.diff.trustB.cc <- lm(Dif.trustB ~ CorSour+Age + PreTrustB  + TrustMed + Lit + Exp,
                           data=treated.data)
```

```{r message=FALSE}
stargazer(model.diff.trustB,model.diff.trustB.cc,model.diff.trustB.c,
          dep.var.labels = "Post-Pre Difference in Trust", covariate.labels = 
            c("Pre-treatment Trust","Media Trust","Media Literacy","Media Experience",
              "Identify Source","Age","Female","Education","Income","Constant")
          ,type="text")
```

### Appendix A Figure 1

```{r}
new.data.c <- data.frame(PreTrustB = 4, CorSour = 1, Age = median(treated.data$Age,na.rm=T), 
                         Female = median(treated.data$Female,na.rm=T), 
                         Edu = median(treated.data$Edu,na.rm=T), 
                         Income = median(treated.data$Income,na.rm=T), 
                         TrustMed = median(treated.data$TrustMed,na.rm=T), 
                         Lit = median(treated.data$Lit,na.rm=T), Exp = seq(5,25))

new.data.nc <- data.frame(PreTrustB = 4, CorSour = 0, Age = median(treated.data$Age,na.rm=T), 
                          Female = median(treated.data$Female,na.rm=T), 
                          Edu = median(treated.data$Edu,na.rm=T), 
                          Income = median(treated.data$Income,na.rm=T), 
                          TrustMed = median(treated.data$TrustMed,na.rm=T), 
                          Lit = median(treated.data$Lit,na.rm=T), Exp = seq(5,25))
```

```{r}
predict.c <- predict(model.diff.trustB.c, new.data.c, interval = "confidence")
predict.nc <- predict(model.diff.trustB.c, new.data.nc, interval = "confidence")
```

```{r}
plot(1,1, xlim=c(5,25), ylim=c(-3,0.5),type="n",ylab="Predicted Post-Pre Change in Trust",
     xlab="Media Experience",main="",bty="n")
lines(x= new.data.nc$Exp, y = predict.nc[,1], type = 'l',col = "darkblue", lwd=2)
lines(x= new.data.c$Exp, y = predict.c[,1], type = 'l', col='firebrick2', lwd=2)
#abline(h=0,col = "darkgray", lty = 2,lwd=1.5)

lines(x= new.data.nc$Exp, y = predict.nc[,2], type = 'l',col = "darkblue", lty=2)
lines(x= new.data.nc$Exp, y = predict.nc[,3], type = 'l',col = "darkblue", lty=2)

lines(x= new.data.c$Exp, y = predict.c[,2], type = 'l', col='firebrick2', lty=2)
lines(x= new.data.c$Exp, y = predict.c[,3], type = 'l', col='firebrick2', lty=2)
legend("bottomright",c("Identify Source Correctly", "Identify Source Incorrectly"),
       col = c("firebrick2","darkblue"), lty = 1, bty = "n", lwd=2)
```

### Appendix A Table 7

```{r}
exp.wapo <- PNA.data[which(PNA.data$G.Wapo==1 | PNA.data$G.CDw==1),]
exp.tele <- PNA.data[which(PNA.data$G.Tele==1 | PNA.data$G.CDt==1),]
```



```{r}
#Create a dummy variable with 1 indicating that a Wapo group respondent perceived 
#the source of the China Watch article as Wapo
exp.wapo$p.wapo <- replace(exp.wapo$G.Wapo,which(exp.wapo$CorSour==1),0)
#Create a dummy variable with 1 indicating that a Wapo group respondent perceived 
#the source of the China Watch article as China Daily
exp.wapo$p.cd <- replace(exp.wapo$G.Wapo,which(exp.wapo$CorSour==0),1)

#Create a dummy variable with 1 indicating that a Telegraph group respondent perceived 
#the source of the China Watch article as Telegraph
exp.tele$p.tele <- replace(exp.tele$G.Tele,which(exp.tele$CorSour==1),0)
#Create a dummy variable with 1 indicating that a Telegraph group respondent perceived 
#the source of the China Watch article as China Daily
exp.tele$p.cd <- replace(exp.tele$G.Tele,which(exp.tele$CorSour==0),0)

```

```{r}
model.conv.w <- polr(as.ordered(exp.wapo$Conv) ~ Tr.WapoA + Age + Edu + Exp + Lit 
                     + TrustMed + Income + Female , data = exp.wapo, method="logistic")

model.conv.wp <- polr(as.ordered(exp.wapo$Conv) ~ p.wapo + p.cd  + Age + Edu + Exp + Lit 
                      + TrustMed + Income + Female , data = exp.wapo, method="logistic")


model.conv.t <- polr(as.ordered(exp.tele$Conv) ~ Tr.TeleA + Age + Edu + Exp + Lit 
                     + TrustMed + Income + Female, data = exp.tele, method="logistic")

model.conv.tp <- polr(as.ordered(exp.tele$Conv) ~ p.tele + p.cd + Age + Edu + Exp + Lit 
                      + TrustMed + Income + Female, data = exp.tele, method="logistic")
```


```{r message=FALSE}
stargazer(model.conv.w, model.conv.t, model.conv.wp,model.conv.tp,
          covariate.labels = c("Washington Post China Watch","Telegraph China Watch",
              "Perceived as Washington Post","Perceived as Telegraph",
              "Perceived as China Daily","Age","Education","Media Experience","Media Literacy",
              "Media Trust","Income","Female"),type="text")
```
